chore(scripts): drop ad-hoc verify harnesses, move dataset converters to datasets/ - #284
Merged
Conversation
…into datasets/ No-unenforced-test-tree policy (#99/#267): drop the seven ep_verify/ harnesses (#158) and verify_refl_kl_batching.py (#210) -- recoverable from git history -- and move the two dataset converters next to the data they produce (datasets/video_r1_260k/, datasets/daily_omni_av/), matching the datasets/<name>/ convention and the recipes' default data paths. Also fix the daily-omni docstring example, which invoked the video-r1 script by mistake. scripts/ now holds only the check-recipe-targets CI guard.
haonan3
requested review from
CjhHa1,
KemingWu,
celve and
leviking98z-rgb
as code owners
July 31, 2026 04:45
leviking98z-rgb
approved these changes
Jul 31, 2026
Collaborator
|
LGTM |
This was referenced Jul 31, 2026
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Repo policy since #99/#267 is no unenforced test tree — yet
scripts/was quietly regrowing one: #158 left sevenep_verify/torchrun harnesses and #210 a KL-batching regression script. This PR:Deletes
scripts/ep_verify/(7 files) andscripts/verify_refl_kl_batching.py. They were verification evidence for their PRs; they remain recoverable from git history (feat(veomni-ep): expert parallelism for the VeOmni FSDP2 training backend #158 / feat(experimental): add WAN ReFL as a self-contained training package #210) if anyone needs to re-run them.Moves the two dataset converters next to the data they produce, following the existing
datasets/<name>/convention (cf.datasets/pickscore/prpocess.py):scripts/convert_video_r1_260k_to_unirl.py→datasets/video_r1_260k/scripts/convert_daily_omni_dataset_format_to_unirl.py→datasets/daily_omni_av/The qwen3-omni recipes' default
data_pathalready points into exactly these directories, so converter output now lands beside the converter.Fixes a copy-paste bug in the daily-omni converter docstring (its example invoked the video-r1 script) and updates self-referencing paths.
scripts/now holds onlycheck_recipe_targets.py— the pre-commit/CI guard, untouched here (#283 is reworking it separately).Related Issue
N/A
Test Plan
python scripts/check_recipe_targets.py→check-recipe-targets: 2310 unirl _target_ paths resolve.python -m py_compileon both moved converters → OKruff check+ruff format --checkon the moved files → cleangit grep "scripts/convert_\|scripts/ep_verify\|scripts/verify_refl"→ no references anywhere (the removed/moved files were referenced nowhere outsidescripts/; no CI workflow touches them)Compatibility / Risk
scripts/nordatasets/is packaged (wheel includesunirl.*only).scripts/verify_refl_kl_batching.pyas an addition — merging them after this PR without a rebase would resurrect the file. Please drop it from those branches when rebasing.scripts/check_recipe_targets.py; no overlap.Reviewer Notes
AI-assisted (Claude Code); the submitter reviewed the full diff. Duplicate-work check: scanned all open PRs touching
scripts/(#279, #280, #283) — none performs this cleanup; the only interaction is the resurrection risk noted above.Checklist